Natural language processing and machine learning for personalized tasks experience

ABSTRACT

In non-limiting examples of the present disclosure, systems, methods and devices for assisting with task completion are provided. A natural language input may be received. A natural language processing engine may be applied to the natural language input. A primary task associated with the natural language input may be identified. A plurality of subtasks for completing the primary task may be identified from the natural language input. A determination may be made from the natural language input that the primary task or one of the plurality of subtasks is more important than other tasks. The primary task and the plurality of subtasks may be added to a list of tasks in a task completion application. An indication of importance may be associated in the task completion application in association with the task or subtask that is determined to be more important.

BACKGROUND

As computers have become ubiquitous in everyday life, so to has theiruse in making users' lives more productive. To-do lists are one of thekey tools that experts identify as being integral in being productiveand accomplishing goals. However, because to-do lists on computers aregenerally populated via manual input, inputting tasks can becounter-productive to the goal of being more efficient with users' time.Automation of task population is desired. However, it is difficult toaccurately capture the importance of tasks, organization of those tasks,and the identification of shortest paths for task completion.

It is with respect to this general technical environment that aspects ofthe present technology disclosed herein have been contemplated.Furthermore, although a general environment has been discussed, itshould be understood that the examples described herein should not belimited to the general environment identified in the background.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription section. This summary is not intended to identify keyfeatures or essential features of the claimed subject matter, nor is itintended to be used as an aid in determining the scope of the claimedsubject matter. Additional aspects, features, and/or advantages ofexamples will be set forth in part in the description which follows and,in part, will be apparent from the description or may be learned bypractice of the disclosure.

Non-limiting examples of the present disclosure describe systems,methods and devices for assisting with task completion. A taskcompletion application and/or a service associated with the taskcompletion application may automatically import tasks that have beenidentified from natural language inputs received from other applications(e.g., email applications, instant messaging applications, etc.) and/orfrom natural language inputs provided directly to the task completionapplication. In some examples, a task or task intent associated with anatural language input may be identified via application of one or morenatural language processing models to the natural language input. Inadditional examples, one of the natural language processing modelsapplied to an input may identify whether there are subtasks associatedwith the task. Any identified tasks and/or subtasks may then be importedinto a relevant task list in the task completion application. Inadditional examples, a machine learning model may be applied to thenatural language inputs to identify relative importance of identifiedtasks based on input characteristics that are specific to each user.Tasks that are identified as being important and/or reminders associatedwith those tasks can then be surfaced in the task completion applicationin a manner that efficiently triages the important tasks.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting and non-exhaustive examples are described with reference tothe following figures:

FIG. 1 is a schematic diagram illustrating an example distributedcomputing environment for assisting with task completion utilizingartificial intelligence in association with a task completionapplication.

FIG. 2 illustrates the utilization of a natural language processingengine to identify and tag distinct pieces of a natural language inputthat are related to task completion.

FIG. 3 illustrates the identification of a calendar date element relatedto task completion from a natural language input and an associatedsurfacing of that identification via a task completion application.

FIG. 4 illustrates the identification of a temporal element related totask completion from a natural language input and an associatedsurfacing of that identification via a task completion application.

FIG. 5 illustrates the identification of user importance in relation toa natural language input and an associated surfacing of thatidentification via a task completion application in a manner that isnative to the task completion application.

FIG. 6 is an exemplary method for assisting with task completion.

FIGS. 7 and 8 are simplified diagrams of a mobile computing device withwhich aspects of the disclosure may be practiced.

FIG. 9 is a block diagram illustrating example physical components of acomputing device with which aspects of the disclosure may be practiced.

FIG. 10 is a simplified block diagram of a distributed computing systemin which aspects of the present disclosure may be practiced.

DETAILED DESCRIPTION

Various embodiments will be described in detail with reference to thedrawings, wherein like reference numerals represent like parts andassemblies throughout the several views. Reference to variousembodiments does not limit the scope of the claims attached hereto.Additionally, any examples set forth in this specification are notintended to be limiting and merely set forth some of the many possibleembodiments for the appended claims.

Examples of the disclosure provide systems, methods, and devices forassisting with task completion. Task completion is made easier byproviding mechanisms for automatically populating a to-do list withtasks and subtasks that have been identified via natural languageprocessing and/or by identifying important tasks included in naturallanguage inputs so that they can be included in a task completionapplication in a different manner than more unimportant tasks. The moreimportant tasks and/or reminders associated with the more importanttasks can be surfaced in a manner that makes their completion moreefficient and effective than if they were simply included as normaltasks in a to-do list.

According to examples, a primary task and/or one or more subtasks of aprimary task may be identified from a natural language input. In someexamples, the natural language input may be received directly via aninput into a task completion application. In other examples, the naturallanguage input may be provided to a different application (e.g., aninstant messaging application, an email application, a digitalassistant, etc.) and subsequently passed to a task completionapplication. For example, a user may utilize privacy settings associatedwith the task completion application and/or one or more of the otherapplications and specifically allow the task completion application toretrieve messages from those other applications, or allow the otherapplications to push messages to the task completion application. Oncethe natural language input is received, it may be processed to identifyone or more tasks and/or subtasks. The processing of the naturallanguage input may comprise application of one or more natural languageprocessing models to the natural language input. In some examples, afirst layer of processing applied to the natural language input maycomprise application of a relatively inexpensive processing model to thenatural language input to identify whether the natural language inputincludes a task (i.e., identify a task intent associated with thenatural language input). If the natural language input is determined toinclude a task, a second layer of processing may be applied to thenatural language input to identify one or more attributes associatedwith the task and/or to identify one or more subtasks included in thenatural language input.

In still additional examples, one or more machine learning models may beapplied to the natural language input and/or elements of the naturallanguage input to identify whether a user associates a relatively higherlevel of importance with a task or subtask included therein whencompared with other tasks and/or subtasks. For example, a machinelearning model that has been trained on past user inputs (from the userthat authored the natural language input and/or from other users) may beapplied to a natural language input to determine whether certaincharacteristics of the natural language input indicate a higher level ofimportance related to an identified task and/or subtask. Examples ofthese characteristics/elements include: underlining of tasks, bolding oftasks, exclamation marks following tasks, starring of tasks, circling oftasks, and highlighting tasks. Other characteristics/elements associatedwith task importance may be determined and identified via application ofone or more machine learning models based on other patterns identifiedin natural language inputs generated by users.

The systems, methods, and devices described herein provide technicaladvantages for assisting with task completion. Processing resources(i.e., CPU cycles) associated with populating task lists may be reducedvia application of the mechanisms described herein by allowing a taskcompletion application to automatically identify tasks and subtasks fromnatural language inputs from various applications and pull them into newor existing task lists rather than making a user open each of thoseother applications separately, copy any tasks included in messages ofthose applications, and paste them into relevant task lists in the taskcompletion application. Processing resources are further reduced viaapplication of a two-layered processing model whereby natural languageinputs are first processed with a “cheap” processing model to identifywhether those inputs include a task, and only when a task is positivelyidentified, processing the corresponding inputs with a second layer ofone or more “expensive” processing models to identify attributesassociated with an identified task. Efficiency and user experience arealso improved upon by the examples described herein by automaticallyidentifying relatively important tasks included in natural languageinputs. That is, rather than making users manually specify which tasksare important to them, the machine learning models described herein cananalyze the language already generated by users and automaticallyidentify tasks that are important to them. Those important tasks and/orreminders associated with those important tasks can then be surfaced ina task completion application in a manner that makes triaging of tasksrelative to their importance more efficient.

FIG. 1 is a schematic diagram illustrating an example distributedcomputing environment 100 for assisting with task completion utilizingartificial intelligence in association with a task completionapplication. Distributed computing environment 100 includes naturallanguage input sub-environment 102, input processing sub-environment 112and task completion output sub-environment 114. Any and all of thedevices shown in FIG. 1 may communicate with one another via a wired orwireless network.

Natural language input sub-environment 102 includes computing device 104on which a user (“TEAM LEADER”) has drafted email 106. The email 106from [TEAM LEADER] to [TEAM MEMBERS] has “Project A” in the subjectfield, and the body states: “Hi [TEAM]—we have the following Project Atasks to complete before next week: (1) [ITEM A]; (2) [ITEM B]; (3)[ITEM C]. Regards, [TEAM LEADER].” Although this message is illustratedas being composed as an email, in other examples the systems, methodsand devices described herein may be applied to other natural languageinput types (e.g., text message, direct task completion applicationinput, digital assistant input, a speech-to-text input, etc.). In thisexample, a user associated with the message (e.g., the team leader orone of the team members) has a task completion application running onher mobile device (i.e., mobile computing deice 114) as illustrated bytask completion output sub-environment 114. In some examples, the taskcompletion application may be executed entirely on mobile computingdevice 116. In other examples, the task completion application may beexecuted entirely on one or more remote computing devices, such as aserver computing device as part of a task completion service thatoperates in the cloud. In still other examples, the task completionapplication may be executed partially on mobile computing device 116 andpartially on one or more remote computing devices.

A user associated with the task completion application running on mobilecomputing device 116 may authorize the task completion application toretrieve messages associated with one or more other applications and/orservices that the user is associated with. For example, the userassociated with the task completion application may use a single usercredential to access an application suite including one or more of: thetask completion application, an email application (e.g., the emailapplication utilized to send or receive email 106), a direct messagingapplication, a meeting application, a notes application, a presentationapplication, etc. The user may either authorize the task completionapplication to proactively retrieve messages from those otherapplications and/or the user may authorize those other applications toautomatically send their associated messages to the task completionapplication.

Once the task completion application and/or task completion applicationservice receives a message such as email 106, it may process thatmessage with a natural language processing engine and/or one or moremachine learning models. In some examples, the task completionapplication may first apply a first, coarse, natural language processingengine to the message to determine whether the message includes a task.The coarse natural language processing engine may be coarse in that itsprocessing costs are relatively low. In some examples, once the coarsenatural language processing engine has positively determine that thereis a task included in the message, a second, finer, natural languageprocessing engine may be applied to one or more fields of the message(e.g., “subject” field, “from” field, “to” field, “cc” field, bodyfield, etc.). The finer natural language processing engine may requiremore processing resources than the coarse natural language processingengine. However, the finer natural language processing engine may beutilized to identify additional attributes of the message, including oneor more of: what user(s) are responsible for completing the task;whether there are other users associated with completion of the task; adue date associated with the task; a time that the task is supposed tostart and/or be started; and/or whether the task has one or moresubtasks associated with it.

In some examples, the additional attributes identified via applicationof the finer natural language processing engine may be utilized by thetask completion application in combination with one or more otherresources to make task completion more efficient. For example, if thefiner natural language processing engine identifies a time (e.g., 10:30am) associated with a task (e.g., “call mom”), the task completionapplication may prompt an associated user to complete the task (callmom) utilizing specific information tailored based on that information(e.g., call mom at her work number rather than at her home number as sheis likely at work at that time). Thus, the task completion applicationmay surface useful content from the other resources (e.g., contactsapplication, calendar application, etc.) according to the informationobtained via application of the finer natural language processingengine.

In examples where the coarse natural language processing engine isutilized to identify attributes of the message (e.g., persons, places,things and/or times relevant to task completion), the finer naturallanguage processing engine may be utilized to weed out false positivesfrom those attributes. That is in some examples, the more cost intensivelanguage processing engine may identify that attributes are related totask completion to a higher degree of certainty than the less costintensive natural language processing engine.

According to some examples, the coarse level processing may compriseprocessing of a received natural language input utilizing a hierarchicalattention model, and the finer level processing may comprise aconditional random field model, with long short-term feature extractors.In additional examples, one or more of the following may be applied to areceived natural language input in performing coarse and/or fine-grainedprocessing of a natural language input to assist with task completion:topic detection using clustering, hidden Markov models, maximum entropyMarkov models, support vector machine models, decision tree models, deepneural network models, general sequence-to-sequence models (e.g.,conditional probability recurrent neural networks, transformernetworks), generative models, recurrent neural networks for featureextractors (e.g., long short-term memory models, GRU models), deepneural network models for word-by-word classification, and latentvariable graphical models). The application of one or both of thesenatural language processing models is illustrated by language processingelement 110 in input processing sub-environment 112.

In this example, the language processing engines identify that there isa primary task associated with email 106 (i.e., “Project A”), and thatthere are three subtasks associated with email 106 (i.e., [ITEM A],[ITEM B], [ITEM C]). In some examples, the subtasks may be identified assubtasks based on being included in a numbered and/or bulleted list. Inother examples, the subtasks may be identified as subtasks based ontheir values being separated by delimiters in a string. In additionalexamples, the subtasks may be identified as subtasks based on acombination of being identified as members of a string separated bydelimiters and being included in a numbered and/or bulleted list.

In examples, the task completion application may have differentmechanisms for surfacing and/or providing information associated withtasks that are deemed “important” by users and tasks that are not deemed“important” by users. For example, the task completion application maydisplay the important tasks in a more prominent manner than the lessimportant tasks, provide more frequent and/or more noticeable remindersto complete the important tasks, etc. In some examples, thedetermination of whether tasks are important to a user may beautomatically determined via analysis of a natural language input suchas email 106. This is illustrated by unsupervised machine learningelement 108. Specifically, one or more machine learning models may beapplied to a user's natural language input to determine whether a taskand/or a subtask included therein is likely more important than othertasks or subtasks included in the natural language input and/or othertasks that have been previously added to the user's task list in thetask completion application. A machine learning model may be trained onuser data for a specific user, and/or data from other users, todetermine whether certain identified tasks and/or subtasks are importantbased on various inputs, symbols and/or highlights around them. In theillustrated example, email 106 includes an underlining of “[ITEM C]”,and that underlining has been identified as indicating for the specificuser in question that the subtask corresponding to [ITEM C] is thereforemore important than the other subtasks included in email 106. Themachine learning model may identify task importance by other featuresassociated with tasks and subtasks (e.g., bolding, highlighting,starring, all caps, etc.). In additional examples, the machine learningmodel may utilize a user feedback loop such that when it correctlyidentifies important tasks for a user, the user may positively indicatethat correct identification, and when it incorrectly identifies tasks asbeing important, the user may negatively indicate that incorrectidentification. The machine learning model may then take that feedbackinto account when it makes subsequent determinations of task and subtaskimportance.

As illustrated by the user interface on mobile computing device 116, theidentified task (Project A) from email 106 has been imported into theuser's task completion application, and specifically the user's WorkTo-Dos. This is illustrated by Project A element 120. The taskcompletion application has also identified via one or more of thenatural language processing engines that there is a due date forcompleting Project A of Monday, January 10, and that due date has beenadded for the task in the task completion application as well. Thesubtasks that have been identified from email 106 have also been addedto the user's task completion application under their correspondingprimary task. The subtask for [ITEM C] has a corresponding element 122on the user interface that is associated with a star, which indicatesthat it is more important than the other subtasks. That element 122 isalso displayed above element 124 corresponding to the [ITEM A] subtaskand element 126 corresponding to the [ITEM B] subtask.

FIG. 2 illustrates an environment 200 for the utilization of a naturallanguage processing engine to identify and tag distinct pieces of anatural language input that are related to task completion. Email 206has been drafted on computing device 204 in natural language inputsub-environment 202. That email is the same message that was discussedin reference to FIG. 1.

In this example, a first level natural language processing engine (e.g.,a coarse processing engine) may analyze email 206 and make adetermination of whether language in that email and/or languageassociated with one or more fields of that email indicate that there isa primary task included in email 206. This processing is illustrated byprimary task identification element 208. In other examples, theidentification of a primary task may be made at natural languageprocess/machine learning element 212. Thus, in some examples, only asingle natural language processing model may be applied to the naturallanguage input. However, by first performing a coarse processing of amessage as illustrated by primary task identification element 208, asmaller amount of processing resources can be expended by only passingmessages that are positively identified as including a primary task foradditional, more cost-intensive processing.

In this example, once identification of the primary task has been made(i.e., identification of “Project A”), the task completion applicationpasses the natural language input 210 from the body of email 206 forfiner processing via natural language processing/machine learningelement 212. As shown by attribute identification sub-environment 214,the natural language processing engine applied at natural languageprocessing/machine learning element 212 has identified “[TEAM]” 216 fromthe text as the “Who” 224 the task relates to, “Project A tasks” 218 asthe “What” 226 the primary task is, “next week” 220 as the “When” 228the task needs to be completed, and the subtasks “(1) [ITEM A]; (2)[ITEM B]; (3) [ITEM C]” as the “What/How” 230 the primary task is goingto be completed. The machine learning applied at natural languageprocessing/machine learning element 212 may have also identified theunderlining of “[ITEM C]” as being more important than the othersubtasks included in email 206. For example, a machine learning modelapplied to past user inputs for the user that drafted email 206 mayindicate that the user indicates that tasks/sub-tasks are important byunderlining them.

FIG. 3 illustrates the identification of a calendar date element relatedto task completion from a natural language input and an associatedsurfacing of that identification via a task completion application. Atask completion application is running and currently open on mobilecomputing device 300. In this example, the user has navigated to a “HomeTo-Dos” page of the task-completion application, where there are twoto-dos/tasks that have previously been added to the “Home To-Dos” list.The previously-added to-dos are “Oil top of windows and doors” and “Calldoctor”. The call doctor to-do is indicated as being important by thestar on the right side of that to-do element. The call doctor to-do alsohas a reminder set for it of Monday, June 25.

A user of the to-do application has initiated the entry of a new hometo-do as indicated by new to-do element 302. Specifically, the user hastyped the following text/natural language input into new to-do element302: “Buy winter tires for next Saturday today at 9:00 please!” Becausethis text is input directly into the task completion application thereis no need to perform coarse natural language processing on the input todetermine whether it includes a task. Rather, the input can be directlyprocessed with more fine-grained natural language processing to identifyvarious attributes associated with the task (e.g., who, what, when,where, why, how). In this example, the natural language processingidentifies “next Saturday” as relating to a “when” and the taskcompletion application surfaces selectable icon 306 which the user mayselect to add a task event due date for the new task on the datecorresponding to next Saturday (i.e., Saturday, August 4). Additionalattributes that may be identified from the natural language input arediscussed in relation to FIG. 4 and FIG. 5 below.

FIG. 4 illustrates the identification of a temporal element related totask completion from a natural language input and an associatedsurfacing of that identification via a task completion application. Asdiscussed above in relation to FIG. 3, a user has input the naturallanguage input “Buy winter tires for next Saturday today at 9:00please!” into new to-do element 402 in the task completion application.In this example, the natural language processing applied to the inputfurther identifies “today at 9:00” as relating to “when” and the taskcompletion application surfaces selectable icon 406 which the user mayselect to add a task event due date for the new task on the datecorresponding to that time element (i.e., “today”). Additionally,because the natural language processing applied to the input has alsoidentified a specific time on the date, the task completion applicationalso surfaces selectable icon 408, which if selected will set analarm/reminder for 9:00 on the date identified (today).

FIG. 5 illustrates the identification of user importance in relation toa natural language input and an associated surfacing of thatidentification via a task completion application in a manner that isnative to the task completion application. As discussed above inrelation to FIG. 3 and FIG. 4, a user has input the natural languageinput “Buy winter tires for next Saturday today at 9:00 please!” intonew to-do element 502 in the task completion application. In thisexample, a machine learning model applied to the natural language inputhas identified that this user indicates that tasks should be marked asimportant when the user adds an exclamation point to the end of a tasksentence. Thus, the task completion application identifies exclamationpoint 504 as potentially being important to the user and surfacesselectable icon 506 which the user may select to mark the new task asimportant in the task completion application. In some examples, markingthe task as important may move the task to the top of the lists in thehome to-dos, result in more frequent reminders for completing the task,result in the task being surfaced in a more conspicuous manner (e.g.,surfacing on the home screen, including a sound and/or haptic reminder,etc.).

FIG. 6 is an exemplary method 600 for assisting with task completion.The method 600 begins at a start operation and flow moves to operation602.

At operation 602 a text-based natural language input is received. Thetext-based natural language input may be received by a task completionapplication and/or a task completion service associated with a taskcompletion application. In some examples, the natural language input maybe received as a direct input to the task completion application (e.g.,a user types the natural language input directly into the taskcompletion application, a user narrates the natural language input fortranscription directly into the task completion application). In otherexamples, the natural language input may be received from a separateapplication and/or operating system feature (e.g., a text messagingapplication, a notes application, an email application, a digitalassistant, etc.).

From operation 602 flow continues to operation 604 where a naturallanguage processing engine is applied to the natural language input. Thenatural language processing engine may be comprised of one or morelayers and one or more processing models. In some examples, the naturallanguage processing engine may comprise a coarse processing layer whereone or more relatively inexpensive processing models are applied to thenatural language input to determine whether it includes a task and/ortask intent. In additional examples, the natural language processingengine may comprise a fine-grained processing layer where one or morerelatively more expensive processing models are applied to identifyattributes of the natural language input in relation to a task and/ortask intent (e.g., what users are associated with the task, how is thetask to be completed, does the task include subtasks, what are thesubtasks, when is the task to be completed, when is the task to bestarted, etc.). In some examples, applying the natural languageprocessing engine to the natural language input may comprise applying atleast one of the following to the natural language input: topicdetection using clustering, hidden Markov models, maximum entropy Markovmodels, support vector machine models, decision tree models, deep neuralnetwork models, general sequence-to-sequence models (e.g., conditionalprobability recurrent neural networks, transformer networks), generativemodels, recurrent neural networks for feature extractors (e.g., longshort-term memory models, GRU models), deep neural network models forword-by-word classification, and latent variable graphical models.

From operation 604 flow continues to operation 606 where a primary taskassociated with the natural language input is identified from thenatural language input. As described above, the primary task may beidentified based on the application of a coarse processing model. Inother examples, the primary task may be automatically identified basedon the natural language input being received directly from the user bythe task completion application.

From operation 606 flow continues to operation 608 where a plurality ofsubtasks for completing the primary task are identified from the naturallanguage input. In some examples, the plurality of subtasks may beidentified by first identifying that the natural language input includesa string separated by one or more delimiters. In additional examples,the plurality of subtasks may be identified by first identifying thatthe natural language input includes a list (e.g., a numbered list, alettered list, a bulleted list).

From operation 608 flow continues to operation 610 where it isdetermined from the natural language input that the primary task or oneof the plurality of subtasks has a higher degree of importanceassociated with it than each other identified task or subtask. In someexamples this determination may be made via application of a machinelearning model to one or more elements of the natural language input. Insome examples, the machine learning model may identify based on pastuser input that the user that input the natural language inputassociates higher importance with tasks that have certain indicationsassociated with them (e.g., the user associates higher importance withtasks that have an exclamation point at the end of them, a userassociates higher importance with tasks that she has underlined, a userassociates higher importance with tasks that she has bolded, a userassociates higher importance with tasks that she has included initalics, etc.).

From operation 610 flow continues to operation 612 where the primarytask and the plurality of subtasks are added to a list of tasks in atask completion application. In some examples, the tasks may becategorized in the task completion application by type (e.g., work type,home type, personal type, etc.). The type may have been identified basedon one of the natural language processing models applied to the naturallanguage input as discussed above.

From operation 612 flow continues to operation 614 where an indicationof importance is surfaced in the task completion application inassociation with the task or subtask determined to have a higher degreeof importance associated with it. In some examples, the indication ofimportance may be surfaced in a format that is native to the taskcompletion application and that indication may be different from anyindication included in the natural language input. For example, the taskcompletion application may identify tasks and/or subtasks as importantvia a star icon next to each important task (i.e., the native format),and the indication included in the natural language input may be anunderline of the important task or word in the important task, or anexclamation point at the end of an important task included in thenatural language input.

From operation 614 flow continues to an end operation and the method 600ends.

FIGS. 7 and 8 illustrate a mobile computing device 700, for example, amobile telephone, a smart phone, wearable computer (such as smarteyeglasses), a tablet computer, an e-reader, a laptop computer, or otherAR compatible computing device, with which embodiments of the disclosuremay be practiced. With reference to FIG. 7, one aspect of a mobilecomputing device 700 for implementing the aspects is illustrated. In abasic configuration, the mobile computing device 700 is a handheldcomputer having both input elements and output elements. The mobilecomputing device 700 typically includes a display 705 and one or moreinput buttons 710 that allow the user to enter information into themobile computing device 700. The display 705 of the mobile computingdevice 700 may also function as an input device (e.g., a touch screendisplay). If included, an optional side input element 715 allows furtheruser input. The side input element 715 may be a rotary switch, a button,or any other type of manual input element. In alternative aspects,mobile computing device 700 may incorporate more or fewer inputelements. For example, the display 705 may not be a touch screen in someembodiments. In yet another alternative embodiment, the mobile computingdevice 700 is a portable phone system, such as a cellular phone. Themobile computing device 700 may also include an optional keypad 735.Optional keypad 735 may be a physical keypad or a “soft” keypadgenerated on the touch screen display. In various embodiments, theoutput elements include the display 705 for showing a graphical userinterface (GUI), a visual indicator 720 (e.g., a light emitting diode),and/or an audio transducer 725 (e.g., a speaker). In some aspects, themobile computing device 700 incorporates a vibration transducer forproviding the user with tactile feedback. In yet another aspect, themobile computing device 700 incorporates input and/or output ports, suchas an audio input (e.g., a microphone jack), an audio output (e.g., aheadphone jack), and a video output (e.g., a HDMI port) for sendingsignals to or receiving signals from an external device.

FIG. 8 is a block diagram illustrating the architecture of one aspect ofa mobile computing device. That is, the mobile computing device 800 canincorporate a system (e.g., an architecture) 802 to implement someaspects. In one embodiment, the system 802 is implemented as a “smartphone” capable of running one or more applications (e.g., browser,e-mail, calendaring, contact managers, messaging clients, games, andmedia clients/players). In some aspects, the system 802 is integrated asa computing device, such as an integrated personal digital assistant(PDA) and wireless phone.

One or more application programs 866 may be loaded into the memory 862and run on or in association with the operating system 864. Examples ofthe application programs include phone dialer programs, e-mail programs,personal information management (PIM) programs, word processingprograms, spreadsheet programs, Internet browser programs, messagingprograms, and so forth. The system 802 also includes a non-volatilestorage area 868 within the memory 862. The non-volatile storage area868 may be used to store persistent information that should not be lostif the system 802 is powered down. The application programs 866 may useand store information in the non-volatile storage area 868, such ase-mail or other messages used by an e-mail application, and the like. Asynchronization application (not shown) also resides on the system 802and is programmed to interact with a corresponding synchronizationapplication resident on a host computer to keep the information storedin the non-volatile storage area 868 synchronized with correspondinginformation stored at the host computer. As should be appreciated, otherapplications may be loaded into the memory 862 and run on the mobilecomputing device 800, including instructions for providing and operatinga to-do list application and/or a daily to-do list creation and/oraugmentation service.

The system 802 has a power supply 870, which may be implemented as oneor more batteries. The power supply 870 might further include anexternal power source, such as an AC adapter or a powered docking cradlethat supplements or recharges the batteries.

The system 802 may also include a radio interface layer 872 thatperforms the function of transmitting and receiving radio frequencycommunications. The radio interface layer 872 facilitates wirelessconnectivity between the system 802 and the “outside world,” via acommunications carrier or service provider. Transmissions to and fromthe radio interface layer 872 are conducted under control of theoperating system 864. In other words, communications received by theradio interface layer 872 may be disseminated to the applicationprograms 866 via the operating system 864, and vice versa.

The visual indicator 720 may be used to provide visual notifications,and/or an audio interface 874 may be used for producing audiblenotifications via the audio transducer 725. In the illustratedembodiment, the visual indicator 720 is a light emitting diode (LED) andthe audio transducer 725 is a speaker. These devices may be directlycoupled to the power supply 870 so that when activated, they remain onfor a duration dictated by the notification mechanism even though theprocessor 860 and other components might shut down for conservingbattery power. The LED may be programmed to remain on indefinitely untilthe user takes action to indicate the powered-on status of the device.The audio interface 874 is used to provide audible signals to andreceive audible signals from the user. For example, in addition to beingcoupled to the audio transducer 725, the audio interface 874 may also becoupled to a microphone to receive audible input, such as to facilitatea telephone conversation. In accordance with embodiments of the presentdisclosure, the microphone may also serve as an audio sensor tofacilitate control of notifications, as will be described below. Thesystem 802 may further include a video interface 876 that enables anoperation of an on-board camera 730 to record still images, videostream, and the like.

A mobile computing device 800 implementing the system 802 may haveadditional features or functionality. For example, the mobile computingdevice 800 may also include additional data storage devices (removableand/or non-removable) such as, magnetic disks, optical disks, or tape.Such additional storage is illustrated in FIG. 8 by the non-volatilestorage area 868.

Data/information generated or captured by the mobile computing device800 and stored via the system 802 may be stored locally on the mobilecomputing device 800, as described above, or the data may be stored onany number of storage media that may be accessed by the device via theradio interface layer 872 or via a wired connection between the mobilecomputing device 800 and a separate computing device associated with themobile computing device 800, for example, a server computer in adistributed computing network, such as the Internet. As should beappreciated such data/information may be accessed via the mobilecomputing device 800 via the radio interface layer 872 or via adistributed computing network. Similarly, such data/information may bereadily transferred between computing devices for storage and useaccording to well-known data/information transfer and storage means,including electronic mail and collaborative data/information sharingsystems.

FIG. 9 is a block diagram illustrating physical components (e.g.,hardware) of a computing device 900 with which aspects of the disclosuremay be practiced. The computing device components described below mayhave computer executable instructions for assisting with taskcompletion. In a basic configuration, the computing device 900 mayinclude at least one processing unit 902 and a system memory 904.Depending on the configuration and type of computing device, the systemmemory 904 may comprise, but is not limited to, volatile storage (e.g.,random access memory), non-volatile storage (e.g., read-only memory),flash memory, or any combination of such memories. The system memory 904may include an operating system 905 suitable for running one or moreto-do list programs. The operating system 905, for example, may besuitable for controlling the operation of the computing device 900.Furthermore, embodiments of the disclosure may be practiced inconjunction with a graphics library, other operating systems, or anyother application program and is not limited to any particularapplication or system. This basic configuration is illustrated in FIG. 9by those components within a dashed line 908. The computing device 900may have additional features or functionality. For example, thecomputing device 900 may also include additional data storage devices(removable and/or non-removable) such as, for example, magnetic disks,optical disks, or tape. Such additional storage is illustrated in FIG. 9by a removable storage device 909 and a non-removable storage device910.

As stated above, a number of program modules and data files may bestored in the system memory 904. While executing on the processing unit902, the program modules 906 (e.g., task completion application 920) mayperform processes including, but not limited to, the aspects, asdescribed herein. According to examples, NLP (first layer) engine 911may perform one or more operations associated with applying a relativelyinexpensive (from a processing standpoint) natural language processingmodel to a message to identify whether that message includes a task ortask intent. NLP (second layer) engine 913 may perform one or moreoperations associated with applying a relatively more expensive (from aprocessing standpoint) natural language processing model to a messagethat has been identified as including a task or task intent, to furtheridentify one or more attributes associated with the task or task intent.Importance identification engine 915 may perform one or more operationsassociated with applying a machine learning model to a natural languageinput from a user and identifying whether characteristics of the inputindicate a relatively higher level of importance related to an includedtask compared with other tasks and/or subtasks. Task importancesurfacing engine 917 may perform one or more operations associated withindicating an importance level, in a task completion application, inassociation with important tasks that have been identified as importantbased on application of one or more machine learning models to a naturallanguage input.

Furthermore, embodiments of the disclosure may be practiced in anelectrical circuit comprising discrete electronic elements, packaged orintegrated electronic chips containing logic gates, a circuit utilizinga microprocessor, or on a single chip containing electronic elements ormicroprocessors. For example, embodiments of the disclosure may bepracticed via a system-on-a-chip (SOC) where each or many of thecomponents illustrated in FIG. 9 may be integrated onto a singleintegrated circuit. Such an SOC device may include one or moreprocessing units, graphics units, communications units, systemvirtualization units and various application functionality all of whichare integrated (or “burned”) onto the chip substrate as a singleintegrated circuit. When operating via an SOC, the functionality,described herein, with respect to the capability of client to switchprotocols may be operated via application-specific logic integrated withother components of the computing device 900 on the single integratedcircuit (chip). Embodiments of the disclosure may also be practicedusing other technologies capable of performing logical operations suchas, for example, AND, OR, and NOT, including but not limited tomechanical, optical, fluidic, and quantum technologies. In addition,embodiments of the disclosure may be practiced within a general purposecomputer or in any other circuits or systems.

The computing device 900 may also have one or more input device(s) 912such as a keyboard, a mouse, a pen, a sound or voice input device, atouch or swipe input device, etc. The output device(s) 914 such as adisplay, speakers, a printer, etc. may also be included. Theaforementioned devices are examples and others may be used. Thecomputing device 900 may include one or more communication connections916 allowing communications with other computing devices 950. Examplesof suitable communication connections 916 include, but are not limitedto, radio frequency (RF) transmitter, receiver, and/or transceivercircuitry; universal serial bus (USB), parallel, and/or serial ports.

The term computer readable media as used herein may include computerstorage media. Computer storage media may include volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information, such as computer readableinstructions, data structures, or program modules. The system memory904, the removable storage device 909, and the non-removable storagedevice 910 are all computer storage media examples (e.g., memorystorage). Computer storage media may include RAM, ROM, electricallyerasable read-only memory (EEPROM), flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other opticalstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other article of manufacturewhich can be used to store information and which can be accessed by thecomputing device 900. Any such computer storage media may be part of thecomputing device 900. Computer storage media does not include a carrierwave or other propagated or modulated data signal.

Communication media may be embodied by computer readable instructions,data structures, program modules, or other data in a modulated datasignal, such as a carrier wave or other transport mechanism, andincludes any information delivery media. The term “modulated datasignal” may describe a signal that has one or more characteristics setor changed in such a manner as to encode information in the signal. Byway of example, and not limitation, communication media may includewired media such as a wired network or direct-wired connection, andwireless media such as acoustic, radio frequency (RF), infrared, andother wireless media.

FIG. 10 illustrates one aspect of the architecture of a system forprocessing data received at a computing system from a remote source,such as a personal/general computer 1004, tablet computing device 1006,or mobile computing device 1008, as described above. Content displayedat server device 1002 may be stored in different communication channelsor other storage types. For example, various documents may be storedusing a directory service 1022, a web portal 1024, a mailbox service1026, an instant messaging store 1028, or a social networking site 1030.The program modules 906 may be employed by a client that communicateswith server device 1002, and/or the program modules 906 may be employedby server device 1002. The server device 1002 may provide data to andfrom a client computing device such as a personal/general computer 1004,a tablet computing device 1006 and/or a mobile computing device 1008(e.g., a smart phone) through a network 1015. By way of example, thecomputer system described above with respect to FIGS. 7-9 may beembodied in a personal/general computer 1004, a tablet computing device1006 and/or a mobile computing device 1008 (e.g., a smart phone). Any ofthese embodiments of the computing devices may obtain content from thestore 1016, in addition to receiving graphical data useable to be eitherpre-processed at a graphic-originating system, or post-processed at areceiving computing system.

Aspects of the present disclosure, for example, are described above withreference to block diagrams and/or operational illustrations of methods,systems, and computer program products according to aspects of thedisclosure. The functions/acts noted in the blocks may occur out of theorder as shown in any flowchart. For example, two blocks shown insuccession may in fact be executed substantially concurrently or theblocks may sometimes be executed in the reverse order, depending uponthe functionality/acts involved.

The description and illustration of one or more aspects provided in thisapplication are not intended to limit or restrict the scope of thedisclosure as claimed in any way. The aspects, examples, and detailsprovided in this application are considered sufficient to conveypossession and enable others to make and use the best mode of claimeddisclosure. The claimed disclosure should not be construed as beinglimited to any aspect, example, or detail provided in this application.Regardless of whether shown and described in combination or separately,the various features (both structural and methodological) are intendedto be selectively included or omitted to produce an embodiment with aparticular set of features. Having been provided with the descriptionand illustration of the present disclosure, one skilled in the art mayenvision variations, modifications, and alternate aspects falling withinthe spirit of the broader aspects of the general inventive conceptembodied in this application that do not depart from the broader scopeof the claimed disclosure.

The various embodiments described above are provided by way ofillustration only and should not be construed to limit the claimsattached hereto. Those skilled in the art will readily recognize variousmodifications and changes that may be made without following the exampleembodiments and applications illustrated and described herein, andwithout departing from the true spirit and scope of the followingclaims.

What is claimed is:
 1. A method for assisting with task completion, themethod comprising: receiving a text-based natural language input;applying a natural language processing engine to the natural languageinput; identifying, from the natural language input, a primary taskassociated with the natural language input; identifying, from thenatural language input, a plurality of subtasks for completing theprimary task; determining, from the natural language input, that theprimary task or one of the plurality of subtasks has a higher degree ofimportance associated with it than each other identified task orsubtask; adding the primary task and the plurality of subtasks to a listof tasks in a task completion application; and surfacing, in the taskcompletion application, an indication of importance in association withthe task or subtask determined to have a higher degree of importanceassociated with it.
 2. The method of claim 1, wherein the plurality ofsubtasks is identified by first identifying that the natural languageinput includes a string separated by one or more delimiters.
 3. Themethod of claim 1, wherein the task or subtask that is determined tohave a higher degree of importance associated with it is surfaced, inthe task completion application, above each other identified subtask. 4.The method of claim 1, wherein applying the natural language processingengine to the natural language input comprises applying at least one of:a hierarchical attention model, and a conditional random field model tothe natural language input.
 5. The method of claim 1, whereinidentifying a primary task associated with the natural language inputfurther comprises identifying a date for completing an event included inthe natural language input.
 6. The method of claim 1, wherein thenatural language input is received by the task completion applicationfrom one of: an email application, an instant messaging application, anda digital assistant service.
 7. The method of claim 1, wherein theindication of importance surfaced in association with the task orsubtask determined to have a higher degree of importance associated withit is in a format native to the task completion application, and thatindication is different from any indication included in the naturallanguage input.
 8. The method of claim 1, wherein determining that theprimary task or one of the plurality of subtasks has a higher degree ofimportance associated with it comprises applying a machine learningmodel to a plurality of natural language inputs from a user thatgenerated the natural language input.
 9. A system for assisting withtask completion, comprising: a memory for storing executable programcode; and one or more processors, functionally coupled to the memory,the one or more processors being responsive to computer-executableinstructions contained in the program code and operative to: receive atext-based natural language input; apply a natural language processingengine to the natural language input; identify, from the naturallanguage input, a primary task associated with the natural languageinput; identify a plurality of subtasks for completing the primary task;determine, from the natural language input, that the primary task or oneof the plurality of subtasks has a higher degree of importanceassociated with it than each other identified task or subtask; add theprimary task and the plurality of subtasks to a list of tasks in a taskcompletion application; and surface, in the task completion application,an indication of importance in association with the task or subtaskdetermined to have a higher degree of importance with it.
 10. The systemof claim 9, wherein in identifying the plurality of subtasks, the one ormore processors are further responsive to the computer-executableinstructions contained in the program code and operative to: identifythat the natural language input includes a string separated by one ormore delimiters.
 11. The system of claim 9, wherein the one or moreprocessors are further responsive to the computer-executableinstructions contained in the program code and operative to: surface thetask or subtask that is determined to have a higher degree of importanceassociated with it above each other identified subtask in the taskcompletion application.
 12. The system of claim 9, in applying thenatural language processing engine to the natural language input, theone or more processors are further responsive to the computer-executableinstructions contained in the program code and operative to: apply atleast one of: a hierarchical attention model, and a conditional randomfield model to the natural language input.
 13. The system of claim 9 inidentifying a primary task associated with the natural language input,the one or more processors are further responsive to thecomputer-executable instructions contained in the program code andoperative to: identify a date for completing an event included in thenatural language input.
 14. The system of claim 9, wherein theindication of importance surfaced in association with the task orsubtask determined to have a higher degree of importance associated withit is in a format native to the task completion application, and thatindication is different from any indication included in the naturallanguage input.
 15. The system of claim 9, wherein in determining thatthe primary task or one of the plurality of subtasks has a higher degreeof importance associated with it, the one or more processors are furtherresponsive to the computer-executable instructions contained in theprogram code and operative to: apply a machine learning model to aplurality of natural language inputs from a user that generated thenatural language input.
 16. A computer-readable storage devicecomprising executable instructions that, when executed by one or moreprocessors, assists with task completion, the computer-readable storagedevice including instructions executable by the one or more processorsfor: receiving a text-based natural language input; applying a naturallanguage processing engine to the natural language input; identifying,from the natural language input, a primary task associated with thenatural language input; identifying a plurality of subtasks forcompleting the primary task; determining, from the natural languageinput, that the primary task or one of the plurality of subtasks has ahigher degree of importance associated with it than each otheridentified task or subtask; adding the primary task and the plurality ofsubtasks to a list of tasks in a task completion application; andsurfacing, in the task completion application, an indication ofimportance in association with the task or subtask determined to have ahigher degree of importance with it.
 17. The computer-readable storagedevice of claim 16, wherein in identifying the plurality of subtasks,the instructions are further executable by the one or more processorsfor: identifying that the natural language input includes a stringseparated by one or more delimiters.
 18. The computer-readable storagedevice of claim 16, the instructions are further executable by the oneor more processors for: surfacing the task or subtask that is determinedto have a higher degree of importance associated with it above eachother identified subtask in the task completion application.
 19. Thecomputer-readable storage device of claim 16, wherein the indication ofimportance surfaced in association with the task or subtask determinedto have a higher degree of importance associated with it is surfaced ina format native to the task completion application, and that indicationis different from any indication included in the natural language input.20. The computer-readable storage device of claim 16, wherein indetermining that the primary task or one of the plurality of subtaskshas a higher degree of importance associated with it, the instructionsare further executable by the one or more processors for: applying amachine learning model to a plurality of natural language inputs from auser that generated the natural language input to determine a patternassociated with highlighting the importance of a plurality of tasks.